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O ' Abstract 

<N ' 

In classical urn models, one usually draws one ball with replacement at each time 
unit and then adds one ball of the same colour. Given a weight sequence (wk)km , 
the probability of drawing a ball of a certain colour is proportional to where k 
is the number of balls of this colour. A classical result states that an urn fixates 
on one colour after a finite time if an only if X!o° ^fc 1 < 00 • I n this paper we shall 
study the case when at each time unit we draw with replacement a number d e INI of 
balls and then add d new balls of matching colours. The main goal is to prove that 
the result in the case of maximal interaction generalizes assuming in addition that 
{wk)km is non-decreasing. 



1 Introduction 



^ '■ 1.1 The model 

o\ 

The history of reinforced urn processes starts in 1930, when G. Polya introduced a now 
famous model to study the spread of an epidemic [5] . The reader curious of the multitude 
of different urn processes that have been studied in the last decades should refer to the 
CSj ; 2007 survey by R. Pemantle jl]. The process we are going to study in this paper is both 
a generalisation of Polya's original process and a particular case of a model of interacting 
urns introduced by the author in [2] in 2010 (see also Remark 1). 

Fix d in N the number of balls to be added to the urn at each step. 

Definition 1.1. A sequence (x n ) neNU { } = (r„, #„) neNU {o} e ((N u {0}) 2 ) nu W i s called an 
urn path (with parameter d) if it satisfies: 

• x = (r ,g ) = (0,0) ; 

• r n and g n are non- decreasing ; 

• VneN u {0}, r n + g n = dn or equivalently (r n+ i + g n+ i) - (r n + g n ) = d. 

Here r n and g n should be interpreted respectively as the number of red and green balls in 
the urn. 

Here is an illustration of urn path with d = 5 up to n = 3: 
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Let us now endow the set of all urn paths with a particular dynamics thus defining 
a stochastic process. Fix (u>i)i eN u{o} 6 (R+) NU ^ '' the reinforcement weight sequence and 



define the quantity n(r,g) 



Wr+Wg 



which we shall understand as the probability of drawing 



a red ball among r red and g green balls. Note that n(r,g) + ir(g,r) = 1. For n > 1, define 
the increment of red (resp. green) balls at time n by Ar n = r n -r n _i (resp. Ag n = g n -9n-\)- 
Then the dynamics is defined as follows: X = (R n ,G n ) n£N is a Markov chain with X = 
(0,0) and the transition law 

| P [AR n+1 = a\X n = (r,g)] l {Xn=(r , 9)} = (*)ir(r, g) a n(g, r) d - a l {XnHr , g)} , Va,0 < a < d; 
[ AG n+ i = d - AR n+1 . 

It is clear that X is an urn path with parameter d a.s. In words, AR n+ i (and AG n+ \) 
follows a binomial distribution B(d,n(r, g)) conditionally on X n = (r,g): each of the d 
balls added at time n is independently red with probability 7r(R n ,G n ) and green with 
probability n(G n , R n ). 

For n e N , denote by T n the a-field generated by the n first steps: 

J~n ~ & (^0,^1; • • ■ ,X n ) . 

This model can be linked to multiple particles Reinforced Random Walks. Consider 
the following star shaped graph: 




Suppose that there are d particles on the central vertex and at each step each particle 
jumps over one of the edges with probability proportional to u>j, where i is the number 
of time the edge has been traversed by one of the particles since the begining, and then 
jumps back to the central vertex. This dynamic is equivalent to an urn process in which d 
balls are added to the urn at each step and the balls could be of E different colours where 
E is the number of edges (or the number of leaf). In this paper we will limit ourselves to 
two different colours, that is E = 2: 



Remark 1. This model is equivalent to the Interacting Urn Model [2] with d urns in the 
case when the memory sharing is maximal, that is the correlation probability p = 1. In 
that setting all the d urns always draw their balls in the d urns combined. Therefore, 
Theorem 12.31 answers an open question of [2] . 
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For the proofs it will be convenient to use a finer time indexing in which only one 
ball is added at each step. For this time indexing we shall reserve the superscript k 
and an additional tilde: denote by (X k ) kmu {o} = (Rk,G k ) km u{o} a random urn path with 
parameter d = d = 1 defined by 

P [AR k+1 = 5\ (Xt)tsk = = 7r(nfc+iJd^[fc+iJd)' 57r (^[A ; +iJ [i >nA ; +iJ d ) 1 ^, 

on {(X e )e<k = (xe) e <k} for 6 = or 1; 

AGk+i = 1 - ARk+i, 
for k > 1 where 

[k\ d = max{f e dl : £ < k] = d 

In words, at each time k we add a ball that is drawn in the urn using the configuration 
at time [k\d, which is the last time when the number of balls in the urn was a multiple 
of d. Note that [k\\ - k - 1 (and not k), we prefer to use this notation since it matches 
well the dynamics described above. 

Then the process X = (Rd n , Gd n )n<-N follows the law of the urn process with parameter 
d previously defined. The proof of this fact is left to the reader. 

2 First results 

An interesting hypothesis about the reinforcement weight sequence is the Strong Rein- 
forcement Hypothesis: 

OO 1 

Soo-V— <oo. (SRH) 

The reason why this hypothesis is natural to consider is the following result of Davis [I] . 
Denote by A the event 

(There exists a time n such that after n all the balls 
drawn out of the urn have the same colour 

Then we have the following proposition. 

Proposition 2.1. (Davis, [U) If d=\, then P [A] = 1 if HSRM) is satisfied and P [A] = 
otherwise. 

An elegant proof of this proposition using a continuous time construction of urn pro- 
cesses can be found in pQ or [I]. 

It is therefore natural to wonder whether such a result is true when d > 2. It is 
believable that the following conjecture is true but difficult to prove. 

Conjecture 2.2. For d>2, if kSRR]) is satisfied then P [A] = 1. 

The main result of this paper is the weaker result: 

Theorem 2.3. For d>2, if liSRH]) is satisfied and w is non-decreasing then P [A] = 1. 
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The proof of this theorem will be given in the next section. It is actually an adaptation 
of an argument given by Limic and Tarres in [3] showing that a reinforced random walk 
on some general graph is eventually attracted by a single edge, assuming that the rein- 
forcement sequence satisfies the fISRHj) and additional technical assumptions including 
the non-decreasing case. 

Note that for d > 2 the reciprocal of Conjecture 12.21 is not true since we have the 
following proposition. 

Proposition 2.4. If d > 2, there exist a reinforcement weight sequence which does not 
satisfy liSRH]) but such that A happens almost surely. 



Proof. We construct a counterexample. Fix p > 1 and define for instance (wi)i eN u{o} as 
follows: 

1 if % is a multiple of d; 

p l else. 



W.; 



Obviously this w does not satisfy fISRHI) since there are infinitely many i such that Wi = 1. 
The proof proceeds in two steps: 

i) There exist almost surely infinitely many n > 1 such that both R n and G n are not 
multiple of d. 

ii) For each time n such that R n and G n are not multiple of d, suppose without loss 
of generality that R n > G n . There is a positive probability uniformly bounded away 
from that the urn draws only red balls after time n. 

Lemma [2.41 follows from those two points by Levy 0-1 law. 

First step. For each n such that R n is a multiple of d, the probability for R n +i to be 
also a multiple of d equals the probability that all the balls draw at time n + 1 have the 
same colour. It equals 2 x (l/2) d = 2 x ~ d < 1 (there is 2 choices for the colour and then each 
of the d balls is of this colour with probability n(R n ,G n ) = ir(G n ,R n ) = = 1/2 since 
Wi = 1 when i is a multiple of d). Therefore after geometric one half many steps R m is 
not anymore a multiple of d. This proves the first point. 

Second step. Suppose that R n is not a multiple of d. We use classical expectation 
calculations to find a lower bound uniform in n for the probability to draw always the 
same colour after time n. Let us suppose without loss of generality that R n >G n , so that 
on {R n g?n} u {R n > G n } the probability to draw only red balls after time n is 



P[R n+k = Rn+k-i + d^k^ l\F n ] = E 



+d} 



k=l 



T 



= lim E 



K 



W ^-{Rn+k-Rn + k-l+d} 
fc=l 



T 
■J n 
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by monotone convergence theorem. An then 

K-l 

E [ 1 {R n+K =Rn + K-i+d}\^n+K-l] I~I 1 {R n+ k=R n+ k-i+d} 

k=l 



v[R n+k = R n+k - 1 + d.yk>l\F n ] = lim E 

A->oo 



lim E 

A^oo 



P 



Rn+K-l 



l ^ A-l 



lim E 

A^oo 



> lim E 

A— >-oo 



pRn+K-1 + pG n +K ~ , I 

, (2 A-l 



I" I "K-Rn + fc=-Rn+fc-l 



+d) 



\ 1 + pG n -R n -(K-l)d J 11 1 {^+fc=-R„+fc-i + 

^ 1 + p-(K-l)d J 11 "'-{• R n+fc=^ + fc-l 



77, 



where we use the fact that, on f)k=i {Rn+k = Rn+k-i + G u+ k-i - R n +K-i = G n - R n 
(K - l)d and where the last inequality is due to R n > G n . So by induction 



P[Rn +k = Rn +k -l + d,Vk>l\F n ] > hm ( I + \ K _ 1)d ) E 



A-l 



I" I "H-Rji + fc=-Rn+fc-l 



+d} 



k=l 



T 



i^n^YT^^j =(n iT77 ^j >o. 



This lower bound is positive and uniform in n, so it proves the proposition. 
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3 Proof of Theorem 2.3 



To simplify notations, denote by c k the colour that is drawn in the urn at time k: 



r 




if Rk - Rk-i + l; 
if G k = Gfc-i + 1. 



Then, define the following process: 



Wi 



Rk 1 

E 



E 



The two reasons for which the process N is interesting are on the one hand its relation 
to the event A: 



A ( 



Rk 



oo and Gi 



oo 



N, 
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and on the other hand the fact that it is closely related to the other process 



M k :=Y 
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which is a martingle because 
K[M k -M k - X \F k . x ] 



W R t 



-P [cfc =r| Tk-x] — P [ c fc = g\ F k -i] 



W G 



L*J. 
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and will be very helpful in the proof. Note that the two quantities 



lim N k and lim M k 

exist almost surely and are finite since both iV and M are defined as differences of two 
non- decreasing sequences with a finite upper bound. We denote these two limits by N x 
and Moo respectively. If d = 1 then M = N and in that case one can check that the 
assumption that w is non-decreasing is not needed for this argument to work. 

Thus, to prove Theorem 12.31 we have to prove that + almost surely. 

Let us first give a rough sketch of the proof. We define 



X k := min( J R Wd ,G Wd ) and B k := £ 



1 



W 
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which will be an interesting quantity to compare the variance of iVoo - N k with. A rough 
sketch of the proof of Theorem 12.31 is given by the following picture. 

Step 2. At any of those times k we 
start a new process that has the incre- 
ments of M. With positive probability, 
this process stays above \Ja.B k \2. 

< 2d/w Xk 

Step 3. The difference between the 
two processes is less than 2d/wx k so if 
2d/wx k < \fuB~k\2 it proves Theorem 




Step 1. Either N does not go to or 
there are infinitely many times k such 
that \N k \>^/^B k ~. 



Step 4. If 2d/wx k > \JotB k \2 it means 
that the reinforcement is very strong and 
there is a much simpler direct proof of 
Theorem O 



For the proof to work, we have to choose a for instance equal to (24 + 16dSoo) -2 - We 
will see in the proof how this quantity appears. 

Let us start with the step 1 that will be given by Lemma I37T1 For all k > we define: 



and 



T k := min {£ > jfe : Nf >aB k ) , 
J := {T fe < oo} u {A^ 0} = {iV^ 0} 
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Lemma 3.1. We have 



P[J\?k]>->0. 



In words it means that with a positive probability uniformly bounded away from (the 
value 1/12 has actually no importance), the process N either does not go to 0, or exits 
the interval [-\/aBk, \/aB k ]. 

Proof. For some convenient reason we will focus our analysis on the stopped process N Tk 
instead of N. To prove this lemma we will use two calculations using the variance of N Tk . 
The first one is as follows: 



B k *P[J c \F k ] = E[ljcB k \F k ] 



= E 



< E 



ij° E 



i=X k U>i 



■ — ■ r 



i=k 



J~k 



because on J c , we have T k - oo so any 1/wf from X k to oo appears at leas once in the 
last sum. Then 



B k xP[J c \F k ] < E 



i=k 



Fk 



= E 



= E 



< E 



E (« fe i) 2 - i N I k f - 2N I k ( N l\ - N I k )) 



i=k 



Fk 



oo 

E« fc i) 2 -(A^ 



i=k 



J~k 
Fk 



-2E 



i=k 

+ Ads^aBk, 



J~k 



where the last inequality is due to the upper bound: 



E^«\-7Vf) 



i=k 



Fk 



< E 



. i=k 

< aB k E 



ElflKi-fl 



EK-jvf' 

i=k 



J~k 



< 2ds 00 aB k , 



because in the last sum, each term of the form l/w k appears at most 2d times (d times 
for drawings of red balls and d times for drawings of green balls). Then : 
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B k xP[J c \F k ] < E 



E(^i) 2 -(^ ft ) 2 



i=fc 



+ Ads^aBk 



< E[(N^) 2 -(N^) 2 \^ k ] + 4d Soo aB k 

< E [ (N% - N^f\ T k ] + 2/V fe Tfc E [ (jV£ - 7V?)| JT fe ] + Ad Soo aB k 

< E [ (jV£ - Nl k ) 2 | J- fe ] + (2^^ + 0£) + Ad Soo aB k 

< {Nl k fv[J c \T h ] + E [(JV£ - iV?) 2 lj-l ^ fc ] + B k (4a + 2^+ 4da Soo ) 

< (iV?) 2 P [J e \F k ] + P (2 (^B~ k + ^B~ k ) 2 + 2(/Vf) 2 ) 



So we have: 

P [J c \ F k ] (B k - N 2 k ) < P [J\F k ] (2aB k + A^B k + B k + 2N%) + B k (Aa + 2^ + Ada Soo ) 
Using P [J^Fk] = 1 - P [v/l^fc] this could be rewritten as: 



B k -N 2 <P [J\F k ] (2aB k + A^B k + 3B k + N 2 ) + B k (Aa + 2^ + Adas x ). 
And so, on {TV? < B k /2} we have: 



P[J\F k ]> 



\ - Aa - 2^/a - Adas c 



2a + 4 v /a + 3 + i 

Any a small enough so that the last quantity is in (0, 1) would suffice here. For instance 
with the choice a = (24 + 16dsoo) -2 we obtain P \_J\ T k \ > 1/12 and Lemma [3. II □ 

By Levy 0-1 law, Lemma 13.11 gives that almost surely either + or there are 
infinitely many times k such that |iVfc | > v/ aB ko . It is now time to compare iV with the 
martingale M. More precisely for any k we introduce the process M fc fc ° = M k -M ko + N ko 
for k>ko which is equal to iV at time k - ko and has the increments of M after time k$. 

Let us start with an upper bound for the difference between the increments of N and 

M: 



ko+k 



(M k0+k - M, 


ko+k 


( ku- 


e=ko+i 


{ ™% 


ko+k 


1 {c^=t} 


E 

£=k +l 


R m d 



w, 



G 



ko+k 

- E 



L{c,=t} l{c,=g} 



£=fc +l V W Ri-i 



W 



ko+k -|_ 



W 



Ri- 



E 



{Q=fl} 



Gi. 



-{^=0} 



e= ko +i 



Wf 



mid 



w 



G ; _i 
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Let us find an upper bound for the first term of this sum. The same upper bound 
will hold for the second term. First notice that since w is non-decreasing this is a sum of 
positive numbers so the absolute value is not needed. Then 



ko+k 1/t l/i 00 l/i l/i 



:o+lJ. 
00 

E 



e=k +i w R[ii d w Ri-i e=[k +i\ d +i w R Wd w Ri-i 



Wf> Wi 



d 



-1 



Mfco+iJd+i w Rw 

00 d 1 1 

= E E — — 

00 d 1 1 

= E E- — 

# LfcQ+Urf i=l w ^ 
d 00 1 -1 

= E E — — 

»=l f _ Lfcp+iJd w R d e +i -i 

The last identity is true because the sum is absolutely convergent. Now for each 
z = 1 . . . d, the sum 

00 1 I 

E 



is alternate with the absolute values of its terms non-increasing because for each I: 



1 1 1 
> > 



W Rm W Rde+i-i W R d (e+i) 



since w is non- decreasing. We conclude that 



All 1 
0< £ < 



Then 



e=k 0+ i w R [lld W R l _ 1 W R [n+ i id 



and 



1, Ar 1 d d 2d 2d 2d 

U, i? Ln+1Jd W G ln+ ij d W min(fl [n+1Jd ,G L?l+1 j d ) W *n + i W X„ 

Note that this upper bound does not depend on k so we can write: 
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Given the result of Lemma 13.11 we now wish to compare this upper bound with 
The proof will now bifurcate in two cases: 



B ri 



Case 1. If liminfj 



then iV is very close to M so it will suffice to prove 



that M stays far away from 6. 

Case 2. If liminffc^oo w2 1 B > then the reinforcement is very strong and we will be 

w x k k 

able to give a direct proof that the urn always draws the same colour after a finite time. 
Let us start with the first case. Since liminf^oo w - 2 1 B ^ = then there exist infinitely 

many k such that 

2d \faB~ k 



w x k 



Note that these k are deterministic and Lemma 13.11 holds for each of them. So almost 
surely either N does not go to or there is infinitely many of these k for which there exist 
a time T(k) > k such that iVy^) > a\/B~k. In the latter case let us consider one such k 
and start at time T{k) the martingale 



M, 



T(k) 



M e -M T(k) + N T(k) , W>T(fc). 



Then introduce the stopping timered 

S = mm\e>T(k) :Mf (k) f 



\/aB~ k 



, 3dyJ Bk 



We have : 



6 o 



+ (3d+l)y/B k P 



T(k) 

b - o 



In the last inequality, the term (3d+l) is due to the fact that the martingale is stopped 
at time S and if it goes above the upper limit 3d then its overshoot is at maximum the 
value of the last jump of which is of the form 1/wj for some i > X k and is therefore 



smaller than 
deduce that 



rB~ k . Then, by using P [mJ W > VaB^/2 



-p[; 



Mj (fc) < x/aB^/2 



we 



M 



T{k) 



JBk 
20 



3d + 1 - y/a 
3d + 1 - y/a/2 



< 1. 



This means that with positive probability uniformly bounded away from 0, either 
M T ( fc ) always stays above \J aB k \2 or it goes above 3d\fB k in a finite time. In this last 
case the Tchebytchev inequality shows that there is also a positive probability for the 
martingale to stay above \/aB k /2 eventually. Indeed 
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Var( M oc -M s \F s ) = z[ (M TO - M s f\ T s ] = E 



X;e[(M, +1 -M,) 2 |^] 
le=s 



< 2dB k . 



So we come to the conclusion that with positive probability uniformly bounded away 

< -M- we can conclude 



from we have Ml' fc ' > \faBkj2 > -p- and since 



N..-M, 



T(fc) 



that A^oo ^ 0. This ends the proof in the first case. 

It now remains to prove Theorem 12.31 in the second case. The proof is to compare 
h Corollary 3 in [ 

such that for k > k Q , 



with Corollary 3 in [3J. Since liminffc^oo w2 1 B > 0, there exists e > and a finite time ko 



1 ^ 1 



w 



This implies, for all k > k , that 



1 °° 1 OO OO 1 OO • _ jL , -I 

i >- £ E:WeE:We j " ! 



wf 



i=k 

OO OO 



i=fc j=i w j 



3=k ~J 

. j - A; + 1 e 3 "(£-k + l) 2 

* £ EE^^tE 



w? 2 ft w] 



j=ke=j 

Then using the Cauchy-Scharz inequality, for all k> k$ 
~ 1 _ ~i-Jfe + l 1 



7T 

7! 



(£- k + l) 2 



»7 



E 




2 1 

e 3 w k 



The value of the constant on the RHS does not matter here, what is important is the 
fact that limsup^^ w k Y,7=k ir < 00 • To conclude, let us calculate the probability for any 
k > k that the urn always draws the majority colour after time k. Denote by B this last 
event, then 



P[B]i n (_«!_)'= (n(!. _!!!_))', (nf, 

The last quantity is bounded away from 0, uniformly in k, since 



2wp 



U w t ~ V6V e 3 < °°' 

Therefore, the Levy 0-1 law implies that with probability one the urn always draws 
the same colour after a finite time. This ends the proof of the second case and of Theorem 

[231 
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